<HTML><BODY>
<A NAME="dopy_users_manual">
<H1>DOPY Users Manual</H1>

<A NAME="basic_architecture">
<H2>Basic Architecture</H2>

<P>DOPY consists of a common library and a set of modules for specific
communication protocols (currently tcp and rsh).  This allows communication
details to be abstracted out with minimal impact to applications.  
<P>At the core of the DOPY application interface is the "hub".  One instance of
the <TT>dopy.Hub</TT> class is defined per application.  It is created either
explicitly during the call of the <TT>dopy.init()</TT> function, or implicitly
when some other DOPY function is called that requires it.  
<P>You may obtain a reference to the hub using the <TT>dopy.getHub()</TT>
function.  Use this reference to add server protocols, register objects, and
obtain references to remote objects.  
<P>In a non-threaded server application, it is also necessary to call the hub's
<TT>mainloop()</TT> method while the application is serving.  Use of
<TT>mainloop()</TT> is optional in threaded DOPY applications (but recommended
if the application can be called in non-threaded mode through a command-line
option).  
<P> <TT>mainloop()</TT> will block the application for as long as the hub's
select loop remains active (i.e.  as long as there are server protocols and
clients).  Unfortunately, at this time <TT>mainloop()</TT> will block forever
because there is no way to remove server protocols.  
<P>Every DOPY protocol has a protocol client class (derived from
<TT>dopy.Protocol</TT> and a protocol server class (derived from
<TT>dopy.ProtocolServer</TT>).  A DOPY application becomes a server by adding a
server protocol instance with the protocol module's <TT>makeServer()</TT>
function: 
<PRE>
   
   import dopy.tcp
   
   dopy.tcp.makeServer(9600)
   
</PRE> 
<P>Parameters of the <TT>makeServer()</TT> function vary from protocol to
protocol.  In the case of the <TT>tcp</TT> module, it is a port number.  
<P>Remote objects can be created for any protocol either explicitly (usually by
calling the <TT>remote()</TT> function in the protocol's module) or dynamically
by calling the hub's <TT>getRemote()</TT> method.  For example, the following
two calls are equivalent: 
<PRE>
   import dopy.tcp

   obj = dopy.tcp.remote('localhost', 9600, 'foo')
   obj = dopy.getHub().makeRemoteObject('tcp:localhost:9600:foo')
</PRE> 
<P>Note that even if you use only the second method, you must still import
<TT>dopy.tcp</TT>.  Protocol modules register themselves with the hub when they
are imported, so if <TT>dopy.tcp</TT> is never imported the 'tcp:...' string
will not be recognized.  This may change in a future version of DOPY.  
<A NAME="initializing_your_application">
<H2>Initializing Your Application</H2>

<P>As stated before, it is possible to explicitly initialize the DOPY hub
through the <TT>dopy.init()</TT> function.  This allows you to pass options from
the command line and to specify other switches affecting DOPY.  
<P>In general, for each command line argument, there is also a parameter in the
<TT>dopy.init()</TT> function.  Command line arguments take precedence over
parameter values.  
<P>To use command line arguments with no other parameters, we would do something
like the following: 
<PRE>
   import sys
   import dopy
   
   otherArgs = dopy.init(sys.argv)
</PRE> 
<P>In the case above, <I>otherArgs</I> will be asigned to a list of everything
in the command line not parsed by the DOPY command line processor (excluding
<TT>sys.argv[0]</TT>, the program name).  
<P>It is also possible to initialize DOPY without passing any parameters.  This
should be used in cases where you do not wish to allow the user to pass command
line options in to DOPY.  
<P>And finally, it is also possible to use DOPY without ever calling
<TT>dopy.init()</TT>.  If you choose to go this route, the hub will be
constructed on demand with the default parameters.  
<A NAME="threading_in_dopy">
<H2>Threading in DOPY</H2>

<P>As of version 0.5, DOPY offers multiple threading modes: 
<DL> <DT> <B>non-threaded</B> (THRD_NONE, none) 
<DD> 
<P>Threads are not used.  This mode should be suitable for systems which (for
some reason) still do not support multi-threaded programs.  
<P>When running in non-threaded mode, it is important that all use of the DOPY
interfaces be limited to a single thread.  <P>
<DT> <B>threaded select</B> (THRD_SELECT, select) 
<DD> 
<P>In this mode a single thread is created for the select() loop.  All method
invocations and protocol handling occur within the select thread.  This is not a
very useful mode, but I suppose it could be used by servers that want to
strictly limit the amount of system resources used by their clients.  <P>
<DT> <B>threaded functions</B> (THRD_FUNC, func) 
<DD> 
<P>This is the default mode of operation in a multi-threaded environment.  It is
just like <B>threaded select</B> mode except that when a server receives a
request from a client, a new thread is started to service that request.  The
method invocation occurs in this service thread.  <P>
<DT> <B>threaded communication</B> (THRD_COM, com) 
<DD> 
<P>This is just like <B>threaded functions</B> mode except that in addition to
initiating a thread for every request, a new thread is also initiated for every
connection.  Thus, every client has its own service thread.  
<P>This was the original threading mode of DOPY.  You might want to try it if
you experience problems with <B>threaded functions</B>.  <P>

</DL>

<P>The threading mode can only be selected during initialization.  If command
line processing is used, the "-dopy-thread" option can be used to select it from
the command line.  Alternately, it can be selected using an explicit option in
the <TT>init()</TT> function: 
<PRE>
   dopy.init(sys.argv, threadMode = dopy.THRD_SELECT)
</PRE> 
<P>The above would initialize DOPY in threaded select mode unless an alternate
thread mode were specified on the command line: 
<PRE>
   $ mydopycmd -dopy-thread func
</PRE> 
<P>Thread mode constants and command line thread mode switches are identified
above in parenthesis next to the thread mode name.  
<A NAME="getting_tracebacks_into_the_server_code">
<H2>Getting Tracebacks Into the Server Code</H2>

<P>When an exception is raised from within a remote method, the DOPY server code
catches the exception and passes it back to the client.  On the client side, the
proxy code re-raises the exception object so that it appears to the client as if
the exception were raised seamlessly across the remote method invocation.  
<P>The only problem with this approach is that traceback information (which is
not associated with the exception object itself) gets lost when the exception is
passed back to the client environment.  
<P>To remedy this, the DOPY server stores the traceback information in the
exception in an attribute called "dopy_traceback".  This attribute contains the
traceback information in the form returned by the traceback.extract_tb().  
<P>The information in "dopy_traceback" is automatically appended to the "real"
traceback if you use the errorText() function in the <B>dopy.tb</B> module: 
<PRE>
   import dopy.tb
   
   try:
   
      # call a method on a remote object
      remoteObject.someMethod()
      
   except Exception, ex:
   
      # print out the _full_ traceback and the exception info.
      print dopy.tb.errorText(ex)

</PRE> 
<A NAME="using_dopy_with_rsh">
<H2>Using DOPY With rsh</H2>

<P>It is possible to run dopy over <B>rsh</B>, <B>ssh</B> or any communication
program that uses standard input and output.  The <TT>dopy.rsh</TT> module is
used for this purpose.  On the client side, just create a remote object from the
<B>rsh</B> module instead of the <B>tcp</B> module.  The first parameter is the
command to run, the second is the object key: 
<PRE>
   import dopy
   from dopy import rsh
   foo = dopy.remote('rsh my.host.com "dopyserver 2&gt;err.out"', 'foo')
</PRE> 
<P>On the server side, we just invoke the makeServer() method and wait for the
end of input: 
<PRE>
   import dopy
   from dopy import rsh
   
   rsh.makeServer()
   rsh.wait()
</PRE> 
<P>See the <TT>rshclient</TT> and <TT>rshserver</TT> programs for an example.  
<P>It is also possible to start a dopy rsh server with inetd to serve DOPY tcp
clients.  Simply add a line similar to this to your <TT> /etc/inetd.conf</TT>
file: 
<PRE>
   13477 stream tcp nowait mike /home/mike/w/dopy/myserver myserver
</PRE> 
<P>"13477" is the port number to listen for.  "mike" is the account that it is
run under (I recommend that you use an account other than "root" to minimize the
impact of any security holes that might arise).  "/home/mike/w/dopy/myserver" is
the name of the server program, and "myserver" is the 0th argument.  
<P>Do not write to standard output from within a dopy rsh server or any method
called by it: standard input and output are the communication channels back to
the client.  Do not write to standard error either (unless it has been
redirected) because this is usually merged with standard output.  
<P>Keep in mind that the rsh approach creates a new instance of the server
program with every client connection.  Applications making use of persistence
will need to coordinate access to the persistent objects explicitly.  
<A NAME="the_dopy_naming_service">
<H2>The DOPY Naming Service</H2>

<P>The DOPY naming service is intended to provide the same functionality as a
CORBA naming service through a Python dictionary interface.  The basic classes
of this service are housed in the <TT>dopy.naming</TT> module.  
<P>At this time, there is no dedicated "name server": any server can become a
name server by registering an instance of <B>StandardNamingContext</B>: 
<PRE>

   import dopy
   from dopy.naming import StandardNamingContext

   hub = dopy.getHub()
   nameService = StandardNamingContext()
   hub.addObject('naming', nameService)

</PRE> 
<P>We recommend the convention of using the object key "naming" for the name
service.  
<P>At this time, the naming service is really just a dictionary.  In fact, from
a purely functional point of view, there is no reason why we could not have
replaced the lines in which the <B>StandardNamingContext</B> was created and
registered with the following: 
<PRE>
   hub.addObject('naming', {})
</PRE> 
<P>It is only appropriate to store <B>RemoteObject</B> instances in the name
server: however, there is nothing to prevent you from storing <U>any</U> kind of
object in the name server.  A copy of whatever is stored under a particular key
will be returned to the client, so a naming service can (to some extent) be used
as a storage repository.  An attempt to store an object implementation in the
naming service will produce unexpected results: retrieval of the object will
return a copy of the object, not a proxy.  
<P>This brings up the broader issue of what the naming service really <U>is</U>.
To some extent, the hub's registry is a naming service: it maps keys to object
implementations.  Likewise, the persistence service is a naming service that
mirrors the directory tree.  It seems as though there should be a more generic
way of representing the tree of objects served by a particular server.  At this
point the system is still very much in flux, so there is no reason not to expect
such a thing to evolve.  
<A NAME="the_dopy_persistence_system">
<H2>The DOPY Persistence System</H2>

<P>The <TT>dopy.pos</TT> module is similar to the <TT>dopy.naming</TT> module
described above, only instead of a tree of object <U>references</U>, the
persistence system represents a directory tree full of pickled python object
instances.  
<P>Use of the system is extremely simple: just create an instance of
<B>FileSysDirectory</B> pointed at the root directory of the file system that
you want to publish: 
<PRE>
   import dopy
   from dopy.pos import FileSysDirectory
   
   hub = dopy.getHub()
   pos = FileSysDirectory('/home/mike/etc/pickled', 'pos')
   hub.addObject('pos', pos)

</PRE> 
<P>In the example above, <TT> /home/mike/etc/pickled</TT> is the root directory
of the pickled object tree, and 'pos' is the object key of the persistence
repository (again, we recommend the key 'pos' just for this purpose).  The
object key passed into the <B>FileSysDirectory</B> constructor <U>must</U> be
the same as the key used in the <TT>hub.addObject()</TT> call.  
<P>Clients can store and retrieve copies of objects using the standard list
operators.  To manipulate them remotely, use the <TT>getRemote()</TT> method: 
<PRE>

   # get the persistence repository
   pos = dopy.tcp.remote('somehost.bogus.net', 9600, 'pos')

   # store an object in the repository
   pos['foo'] = Foo()
   
   # get the remote instance and modify it
   remoteFoo = pos.getRemote('foo')
   remoteFoo.setValue(100)
   
   # get a local copy of foo and modify it
   localFoo = pos['foo']
   localFoo.setValue(200)
</PRE> 
<P>Note that in the example above, <TT>localFoo is not remoteFoo</TT>.  The
<TT>remoteFoo</TT> object is a proxy object for the foo instance stored on the
server.  The <TT>localFoo</TT> object is a copy of the object stored on the
server.  When we call the <TT>setValue()</TT> method, we are changing the local
value; this does not effect the remote instance.  
<P>Normal, DOPY does not impose any constraints upon the objects that it
provides access to.  However, in the case of persistence, the system must be
able to tell the object when it is done calling a method so that the object can
save itself.  To deal with this, it is recommended that persistent objects
derive from <TT>dopy.pos.PersistentObject</TT>.  This class implements three
special methods, <TT> dopy_beforeMethod()</TT>, <TT> dopy_afterMethod()</TT> and
<TT> dopy_setPath()</TT>, which are used to automatically write the file after
any method is called and to manage a single instance in memory.  
<P>Users may wish to provide their own versions of these methods to gain greater
control over when objects are written: for example, it is not necessary for
methods that do not modify an object to cause the object to be written.
However, they should still derive from <B>PersistentObject</B> and call the <TT>
_acquire()</TT> and <TT> _release()</TT> methods, which manage the object
instance in a cache for persistent objects.  
<P>The persistence system is smart enough to maintain a single instance of an
object in memory: if two different clients call a method on the same object, it
will not be loaded twice.  However, this system is not smart enough to take into
account things like overlapping directory trees and symbolic links.  For this
reason, we recommend the following: 
<UL>

<LI> 
<P>If you use multiple root-level persistence repositories and one is a subtree
of the other, make sure that the common root is specified in exactly the same
way.  i.e.  don't do this: 
<PRE>
      # assuming current directory is '/home'
      hub.addObject('pos0', FileSysDirectory('mike/pickle', 'pos0'))
      hub.addObject('pos1', FileSysDirectory('/home/mike/pickle/gherkin', 
                                             'pos1'
                                             )
                    )

</PRE> 
<P>In the example above, the system won't be able to recognize that <TT>
pos0/gherkin/BigUn</TT> is the same file as <TT> pos1/BigUn</TT> because the
root is <TT> mike/pickle</TT> in one case and <TT> /home/mike/pickle</TT> in the
other.  
<LI> 
<P>For systems in which file names are not case sensitive, be consistent in your
use of case.  
<LI> 
<P>Do not use symbolic links between two persistence repositories.  
</UL>

<P>In summary, the system epects that a file be known by only one name.  
<P>The object locking mechanism is still rather flimsy in other respects: when
an object is initially loaded, it is given a "use count" of 0.  This use count
is incremented prior to method invocation and decremented after method
invocation.  If the use count is zero after method invocation, the object is
deleted and removed from the cache.  Obviously, this creates a region of
exposure between the point at which the object is loaded and the begining of the
invocation of the method for which it has been loaded.  
<A NAME="special_methods">
<H2>Special Methods</H2>

<P>DOPY objects may have the following special methods: 
<DL> <DT> <TT> dopy_beforeMethod(self, request)</TT> 
<DD> 
<P>Called <U>before</U> a remote method invocation.  If an exception occurs, it
is returned to the caller.  <P>
<DT> <TT> dopy_afterMethod(self, request, response)</TT> 
<DD> 
<P>Called <U>after</U> a remote method invocation.  If an exception occurs, it
is <U>not</U> returned to the caller.  It is ignored.  <P>
<DT> <TT> dopy_setPath(self, pathname)</TT> 
<DD> 
<P>If the object is stored using the persistence service, this method will be
called when the object is loaded so that the source file's path name can be
given to it.  <P>

</DL>

<P>These methods will only be called if they are present.  
<A NAME="reactors">
<H2>Reactors</H2>

<P>It is often desireable (particularly in single-threaded applications) to plug
custom code into an application's select loop.  For example, your application
might have to read information from special pipes or sockets to communicate with
other non-DOPY programs.  To facilitate this, DOPY provides the <B>Reactor</B>
class.  Instances of classes derived from <B>Reactor</B> can be added to the hub
using the <TT>addReactor()</TT> method.  
<P>Reactors provide a standard interface for plugging into the DOPY select loop.

<P>Every reactor class must provide a <TT>fileno()</TT> method, which returns a
handle suitable for inclusion in a <TT>select()</TT> call.  
<P>The <TT>notifyWhenReadable()</TT>, <TT>notifyWhenWritable()</TT>, and
<TT>notifyOnError()</TT> methods must be overriden to return true or false
depending on whether the reactor wants to be notified when its filehandle is in
that state.  The value returned from any of these need not be static: a reactor
may change the events that it is notified of at any time.  These methods are
called every time a state change causes <TT>select()</TT> to become unblocked.  
<P>The <TT>handleRead()</TT>, <TT>handleWrite()</TT> and <TT>handleError()</TT>
methods are what will actually be called when the reactor's file handle becomes
readable, writable or enters an exception state.  These functions will only be
called if the reactor has indicated its interest by returning true from one of
its "notifyWhen" methods.  
<P>When creating reactors, remember that the <TT>handleRead()</TT>,
<TT>handleWrite()</TT> and <TT>handleError()</TT> methods are called
synchronously from the select loop.  They should try to return as quickly as
possible.  If you have to do an extensive amount of processing in response to
one of these events, it is recommended that you spawn it off in another thread,
if possible.  
<A NAME="minicomline_-_a_simple_command_line_reactor">
<H3>MiniComLine - a simple command line reactor</H3>

<P>One <B>Reactor</B> derivative that comes pre-packaged with DOPY is the
<B>MiniComLine</B> class (in <TT>dopy.minicl</TT>).  This class is a handy way
to add a simple python command line to your server (in fact, this is the command
line that you see when you run the <TT>dopyserver</TT> example program).  To use
it, simply add lines such as this to your program: 
<PRE>
   import dopy.minicl
   dopy.getHub().addReactor(dopy.minicl.MiniComLine())
</PRE> 
<A NAME="module_index">
<H2>Module Index</H2>

<P>DOPY is currently comprised of the following modules: 
<UL>

<LI> 
<P> <A HREF="dopy.html">dopy</A> DOPY main functionality 
<LI> 
<P> <A HREF="tcp.html">tcp</A> TCP/IP protocol 
<LI> 
<P> <A HREF="rsh.html">rsh</A> RSH protocol 
<LI> 
<P> <A HREF="naming.html">naming</A> Name service 
<LI> 
<P> <A HREF="pos.html">pos</A> Persistent Object service 
<LI> 
<P> <A HREF="tb.html">tb</A> Functions to help with tracebacks 
<LI> 
<P> <A HREF="minicl.html">minicl</A> Mini command line reactor 
</UL>

<A NAME="copyright_info">
<H2>Copyright Info</H2>

<P>Copyright (C) 1999 Michael A.  Muller 
<P>Permission is granted to use, modify and redistribute this document,
providing that the following conditions are met: 
<UL>

<LI> 
<P>This copyright/licensing notice must remain intact.  
<LI> 
<P>If the code is modified and redistributed, the modifications must include
documentation indicating that the code has been modified.  
<LI> 
<P>The author(s) of this code must be indemnified and held harmless against any
damage resulting from the use of this code.  
</UL>

<P>This code comes with ABSOLUTELY NO WARRANTEE, not even the implied warrantee
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  </BODY></HTML>